The invention relates generally to the transfer of large data files and the control of such transfer in view of available bandwidth and other considerations.
A wide range of systems and schemes have been developed for the transfer of data between organizations. In some of these, transmission is controlled by either the sender or the receiver or by an intermediary in a network between these parties. Problems addressed in this context include both security and appropriate utilization of available bandwidth. Bandwidth limitations may be encountered on both the sender side and a receiver side, as well as at various points therebetween, such as due to limitations of the network, network providers, and so forth. At present, however, many data and file transfer protocols do not adequately regulate transmission, and transmission failures or substantial reduction in transmission rates, even for high-priority transmissions, can occur.
Problems with bandwidth utilization for data and file transmissions are particularly acute when large volumes of data are transferred as compared to the available bandwidth. For example, in media companies, very large files may be transferred on a regular basis, such as from original providers (e.g., news reporters in the field, original content providers, and so forth) and from production and post-production suppliers. The files may include considerable video and audio data which may be extremely extensive, even in compressed formats. In certain environments, such as for news, timely broadcasts, and so forth, such large data files may need to be reliably transmitted on a daily or hourly basis, and throughout the day and night. When the transmissions are made over an internal network, problems with file transfer are more generally easily resolved. However, when files are transferred from external sources to an internal network, such as from content providers to a news organization, media company, or the like, the internal network may be inundated by received transmissions unless appropriate coordination is possible. Essentially, the outside transmitters of messages and large data files may operate on a “fire and forget” approach in which the messages are launched and, regardless of the ability of the internal network to receive and process the data, the transmissions are sent and queued at the destination.
Such problems may ultimately result in significant delays and even the loss of data. In particular, without appropriate coordination and with no effective prioritization, sent messages and files are processed as they are received. Thus, large, low-priority transmissions may saturate available bandwidth to a detriment of higher priority transmissions. Similarly, certain transmissions may be interrupted or aborted due to network transmission and receipt delays.
There is a need, therefore, for improved techniques for managing transfer and receipt of large data files.